Insurance providers value the ability to collect driving information of various users for use in evaluating risk, providing incentives, adjusting premiums, and the like. Although techniques exist to generally capture data from sensors on smartphones and in vehicles, they may not recognize a type of vehicle or mode of transportation associated with data sets (e.g., car vs. train, etc.). While providers desire information relating to the operation of a vehicle by a user, data recorded while the user is a passenger in a car, train, airplane, boat, bicycle, or other mode of transportation may hinder the ability to determine accurate driving information related to the user. Therefore, it is beneficial to recognize the vehicle mode associated with the user during a travel segment.